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1 Introduction 

„ y out of W>- *T»t«« an* J^T**""' Si*? 1MTO 
:ini on it. _«f In. lue* , • 'LT^E.. It 



t«ting on «ay in. ~ • ^.^5 "id~M*iH». It 
tit-rMcMNf off act. on ir—Iia lita to mMnt 

1. ooinot tail ^^ c ^^ t JJ^Z 
.alactad data fro. tha lonMfUn tftttm. 



bohind ttao alxad-ability coneopt og ^^!La1tn 

tor nor. than on* *»"* t * ,d J£*" -SSTVi iaauai 
rooocai for tnli i" that om • e>B> ** , _.^ J T^-* --■»- 

SStr^'su sir Jtss iirar — 

tony othor thing. In .oeiaty. 

but ono a.pact. Hourrar. Mj*. .fgyggl, ^ ... 

por.t. conf.ranca. and £ "gj*,* 0 * * 2^ oca 

Slnar raaort. »o.J * r £ J^'Joint!* 

particular. 

tJ - l<IIM i lk i f ## i that it !■ *o*y is*o r " 

K S£S2 in^ ion ^^^^h Inf o 

fc J2 « -J. ^^-^ 

■hall pra.ant .alactad ra.ult. fro. •~ cl ^ A to find 
car riod out in 1M1. Tha aia of ^''J^'^' lu-lm 
out Uhat a .tati.tic.lly W«jnU*»~ "•j;^g Bt "3.. 
of -tud.nt. and t..char. in orada. J^U^jtui 

.per. 13-1B) of "•"■J 1 " l^IroblM. ^n t&ttlt in 
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the survey evaluation stands is ont out of a con- 
eider* t« number of problem areas. In the context of 
this conference I fttl that It may be just as relevant 
to hNr about such problems as to hear about any solu- 
tions that we alght have to offer. 



2 The lotveglan EFL curriculum »nd t " ' on system 

* Teaching In lor~*glan comprehensive si has since 
1974 been based on the "Mmnsterplen for rutmekolen" 
(Curriculum for primary and lower secondary \»ducation). 
The main educational Innovation Introduced by th*e plan 
was the step amy from a fixed curriculum for all to a 
flexible teaching guide, where Individual schools and 
teachers (at least In principle) are given considerable 
freedom to choose for themselves both what to teach and 
how to teach it. The testlno system, however, is still 
largely centralized. 

In the m, teaching guide a major change conelsted 
ox moving away from explicit tjiowledge of language struc- 
ture and metalanguage to an explicit emphasis on skills 
and a lees explicit emphasis on language as a means of 
communication. The goal of EFL Is stated in the fol- 
lowing three parts (my translation)! 

"The teaching of English Is to aim at 
-giving the students practical language skll] which 
may give them Increased abllltr to establish con- 
tact, orally and In writing 

-forming as firm a basis as possible for continued 
learning of the subject 

•establishing such attitudes toward the language 
that the students develon an interest in Increasing 
their skills and get an Impulse to use the language." 

Based on considerations of communicative needs, oral 
skills are given aore emphasis than written skills. 
Furthermore, of the two written skills reading seems to 

be given priority before writing, which Is partly given 
a secondary role (Slmensen 1983) i 

"The skill of written expression is also to be 
practiced, both as a support of training of the oral 
skills and as s means of expression in itself". 

Given the fact that the teaching guide Is based on 
general goals rather than specific objectives, with con- 
siderable local freedom within the system, one may sus- 
pect that teachers have problems in knowing what to eva- 
luate, which criteria to use. In the national survey 
referred to. the teacher questionnaires contained the 
following qu»stiont 
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Do you think it is emey or elf float to ftmv Ait to 

correct in this grade? (Put a ring wound tt» f iftft 
below the appropriate response alternative)* 

very quite Ojuit* very don't 

diffic. diffic. medium emsv ema> *~ 



rigure 1. Translated Item fro* EPL teacher's esjmetion- 

nalro. 



The anewers o€ 191 aTL tooctwgs in erode t 
played in Table 1 below. In the table anewers arm given 
as percentages for each reeponee alternative. The fre- 
quencies are weighted to counterbalance blames intro- 
duced by disproportionate sanpllng deal en <cf . tvenaen | 
1982a). Furthermore the frequenclee are sat Raited ao that J 
those repondents who anowjred 'don't know' (1.0 per cent 
of the total) or left the item unanswered (3.1 per cent 
ot the total) are kept out of the computations. 

Tab*e 1. Selecting criteria for correcting student per- 
formance. fTL teachers' porceptlone ->t difficulty in 
gL^cl^Jit^^ii^nd tdluiVtd fEWUtnciti m-Ulli 

response alternatives central tendency 

end dispersion 

very *«ry 

diffic. - - »asy 

5 4 3 2 1 m s til 

% % * % * 

02.9 13.5 49.* 12^3 Bfciufl Lfll ■ Hi 

IS. 4 per cent, or about a sixth* of the teachers 
feel that knowing what to correct ie very difficult or 
quite difficult. Tor these teachers the choice of evalu- 
ation criteria mar be said to constitute a problem. 
For the great majority, however, the system sterns to work 
quite well. If we use the response ecale f roe five to 
one to compute an average (median) out of the answers 
given, the result is 2.821. This means that an "average 
answer" is a little toward the 'easy' end of the seals. 
The standard deviation adua the further information that, 
there is some disagreement among teachers on the ques- 
tion! there is some degree of dispersion in the answers 
given* 

The most reasonable conclusion one can drc^ from the 
data awems to be that, after seven yeary of experience 

with mixed-ability teaching, most tearhers ars in fact 
quits satisfied with the freedoe <or burden) of choice 
built into the syitem. The table, in other vords , seems 
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to show that local frasJoa it not as hi. - . 
<*>• .l<rht expected It Co STf" telche?. 

pact to (moving what to corract^ MCheta wlth 

taach^.^^ion M ^. t ^ B r tUd ' nt ' tMnk ° f th * 
tained an it., of tne V?L t \ questionnaire* con- 
student. w r e\.£ed h to EEuaK'tSSSr ?° V '« the 
■onstrated in ►eachar teacher demands, as de- 

presented In Tails 2. " •"•were are 

^.^n W ^ °< ^er 

(nM10). Weighted and adlusted frequencies 



response alternatives 



central tendency 
demands are—- and dl3 P» r «i°n 



aruch 

too 
gr^at 



■uch 

too 

ftaall Md 



% * % % % 



with i'^'s vSi?^; *7 not totaiiy 

teacher d-aands are -urh\ P 5 Cent feei th ** 

areata On^To^r S3 e.s^o feV'^ J2° 

teachers are auch toj lem^n? 1 2 feel that the 

the teachers are a i^m" f™" 1 . P " C<fnt think th *t 
fleet, this dt.^bJtlo"' KmS? 1- ^ "I. 1 *" "I 

the too or eat part of the Iclll «h„ i 8 tow * rd 

3 The ff^rwgian #xaB 

In this section I mKa i i n r « a «nt. , - 

:^oif h ?n"?„r P ^^ 

pondence gtS^T^.^ "e .T.,.?.^ t-^^i 

^'^rts^h^-^rthir^ " ^ 

influences practical teaching Si.e^sen ? 198-2! T^'k" 
for.ulated thta principl „ ve 8tr ^^? if f^ri 1- ! 
discrepancy t-tucm to. . . " there l 8 a 

u«, lU y uini . **" ,n, - , * nd "valuation evaluation 
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I have caoeen to um t^Mteir p e rc e ption* e* t^rti- 
pondence as ay starting point. In tha questionnaire 
tnchiri vtn asked to give their views at to the degree 
of correspondence between the goals and the? ETL mi. I 
find the rciulti Tory Interesting* 

Table 3. Toachar perceptions of tho correspondence 
between tha official goals and tha ETL ran at the end of 
comprehensive school. Nalfhtod and ad jus tod fresjuenclee 

IrrWli — 

respo***e alternatives centrml tendency 

and di spar elm 

very vory 

<|ood bad 

corr. corr. ■ 

* 4 3 2 1 

% % % % % 

op. o 03.fr 4*.a Jft.fl ai*3 5L^M Oil 



It ssems clear that wo doal hars with a roal problem 
of some Intensity, ions of tha teachers think that tho 
corrsspondsncs Is vary good', and only 3.§ par cant 
feel that It Is qui to good'. On tha other hand almost 
ten par cent think tha* the correspondence Is 'vary 
bad' . Tha aedlan Is as low as 2.966 and the low standard 
deviation <.722) indicates a higher degree of COTiceneus 
among the teachers on this question than on previous 
ones. In an effort to understand tnis problem X shall 
first present soae background Information about tha esaa 
systt«* and then supplement this material with more de- 
tailed surrey data. 

English is tha oni> foreign language with a final 
written exam in ftorway. The test takes piece at tha and 
of ninth grade and Is allotted four hour", fifteen 
alnutes for practicalities!. I; officially consists of 
at least four out of a list of eight subtests pi rented 
in the Hant&ook for the school, part III (Hardbok for 
stolen, drl XXI) i 

-comprehension questions on a given text 

-flii-in-the-blanlc < including close) 

-completion of a text where the beginning or end I ts 
been removed 
-free comments on a text 
-essay based on key words 
-picture-based essay 
-essay on a given topic 

According to the regulations each student sits for 
st least one of the following subjects- 
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-EFL 

In pr.ctic. e.ch .tud.nt air. , 
th««e. which Man. that an ! Z f ? r ° nlr of 
g» third of th. countr;.."^:^*"'" *" Wi-t.l, 

"<l.ntUX%t:. -Ct#d ^ '* ch .eeordX, K'lt 

referenced and nor.- referent te 8 t ?£? e#d ' "it.rio" 

rr Jl'4Si52 a of or *• . JU , ln . rg . 

■*"t not JuBt to lln7ul r ; t a £ e '» r «l*te their jud-..' 

of th « "hoof .;. t p n *f to & 

if * ! temchlr >9 9uide. Apart f rol i h,,# otatwl 
of intention, the e««i£rV.re Jo? *f neral d »"ription 

Priory and Lower-.econd.ry^Ltion ^ C0U " CU ^ r 

5 r J5i~ d C i l .c"e d t e ^ 0 " t8 <" «- 
?iven for e.ch .ubt.Jt (ilJt^ ' * number of point. 
the. t0 i.po rtance • relative *o? 

7lven for under^andab^t ^".a! v V' 1 "" -cor"^ 

to -«e „tent .„ t ^ ^-^1" priZclpE. 

P* r < -f th. -arkino '° thp « i t er lor - r.f er.nc^d 
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ror the essays itifiral broad criteria at* suggested* 
The nityi In *he upper two bends of the f It* -point mar- 
king scale (Nf below) should be "good" to WteN H CI 
(2). diction and vocabulary* having fair arrori of 
orthography and verb Inflection. furthermore tba essays 
have to b« in accordance with the wording of tba task 
given. 

The essays in tba aid group are accepted with 
what lower ■tandard as to coherence* diction, 
lary, errors of orthography and verb inflection* 
naive formulations and a relatively abort answer, 
criteria for the low group are more indirect t a 
■hort •■■ay will not receive good marks and an other- 
wise aedlua essay should not be lowered sore than one 
step in the aar Icing if the student has not written the 
•nay part of the exam. For the lowest eark the cri- 
terion is coawunlcatlvei the essay would not be under- 
stood if performed orally. 

By way of contrast, the oral exaa consists of a 
combination taken out of a list of four subtests 
iHandbok for stolen, del III) i 

-reading of a passage from a known text followed by 
conversation about the text froa which the passage 
Is taken 

-rree conversation about everyday topics 

-reading of a short, unknown text followed by compre- 
hension questions 

-conversation about an unknown text read aloud by the 
teacher (the Internal examiner » 

Certain students froa two counties are selected to 
sit for the oral exaa in Engl lib every year. Thl«j neons 
that the proportion of students sitting for the oral 1b 
auch lower than the proportion sitting for the written 
sxaa «cf. T*ble 5 below). j 

There 1b one external examiner for each student, and * 
each team of examiner s evaluates toe performance of some 
20 students. In the aar king regulations the emphas.' i is 
explicitly on the students' ability to carry on a con- 
versation, which is aore Important than her/his pro- 
nunciation and Intonation. The regulations do, here as 
well, have a communicative element. 

Statistics as to the distribution of subtests are 
only available for written exaas. In the following 
section I shall concentrate on the written exaa, supple- 
menting snee statistics and survey data thit are rele- 
vant also for the oral. 

The nor*~ref erenced part of the evaluation criteria 

Is quite indirect. National ntandardiied tests in the 
structuralist -ptychoaetric tradition (Spolskv) have 

since 1962-63 been offered on an optional basis as one ; 

i 
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eleaent guiding the teachers tchimunt urkln „ -n, t . 
■y«te. 1. used by 80-90 per c.nt tht ThSi ^? 

•***- it mt b« assuaed that these testa have 
long-r.nga effects on individual teacher aarkino S£h 
because of their ion, tradition and because o? tE2 

but!d\s JTrii T' The w Mrk « t° nora.li r diStri- 
1-mI. L?f« ° P ^f* " "Particularly good"- and Lg 
is the hotto. aark - little of positive value"). ^ 



Je ! ? ^ 



«* 24% 48% 



2*% «% 



Let us look at the :haracterist ics of the written 
^ 4> Distribution of .uhf.t. in recent EFL written 



-open-endsd coaprshension 

questions 
-true/false iteas 
-te*t -based questions/ 

tniwtri 
-story/dialogue coapletiona 
'vocabulary test 
-clo*e/ fill-m-the~blank 
-transf oraations and 

paraphrases X XX ~ 

-sentence coapletions - ^ ~ 

"^•at-based free coaaents 
-expression of selected 

language functions x 
-letters/ postcards x x ~ 
- tl lft Y U gun qf j-fr t lA k*) x y x ^ 



i960 


'81 


'82 


'83 


84 


X 


X 


X 


X 




X 








X 


X 


X 


X 


X 


XX 










X 


X 




X 


XX 


X 


X 




X 


X 


XX 



ERIC 



th^ ^! dlatrlbutl on of subtests clearly indicate*, 
that the average nu-ber of subtest, is l« r<7 . r tUn four 

9 3 



71 



Tha frequancy of tha fottm «JlU fg • 

eoaaamt? ThT-o-t llk.iy «»l«»tlon 

unarpactad pattern 1. rufoartM In • *T*""^ !*g 
by Siaan.m <1»7»). Cn tha barlr of ^"^ 1 ^* ln ^L (? I^d 
of EFL curricula darelopnent an* «*«• •»- concl"*^ 
that rararal a.pact. of th. mu tr^lltlan -int ba 
quartlonad «ten r.latrd to th* official «rrte«a. On. 
tuch arpact H th. pr.pond.rmM of "• M J*? r * *2wi"*T n 



Sint. "r. only trained to ^mmw glv^ J-^^^* ^ 



quirt ion.. An ln.poctlon of th. •«*- J"^** «g 

und.r rtudy rhovr that It 1. In f.ct thlr ••^ t .* hm * " 
a^pha.l.ad* thlr rubta.t in .11 cm- 
StUnitn r.port Hki th. rtud«.t to •* Mtl ° n * 
that .r. r.l.rant to • qirm tart or rltuat on. 



In BlMnitn'i (1979) rtudy th. «»Plclon ^J* • 
rubt.rtr aay not dlrcrl.lnete wall «"» alro ^'••J**! 
Sh. furthermore rueeertad that th. U*. of ^^W^ 
and dl.cr.t.-polnt t..t. of «ra»*r rhould ba 
In th. parlod under rtudy It appear! that there bar 
not « lncrMit in th. n«bar of ruot.rtr i*trt too 
vrrrionr differing In difficulty (ee. In that on. rub- 
tuit lr guldad ahare a rl.Uar rubtart lr not or that 
th. ■rrkln* of on. rufct.rt taker accuracy Into account 
where th. other dorr not) ara uaed. 

I„ th. P-rlod artar 
har. relied on eetallnguirtle knoeladrje. J*""*" *• 
_t_, f diaerata-oo^nt tartr of eraaaar har. rxin tmma 
ETiS "not'rlE. 1m. It -Tb. Justified to conelu- 
d. that on. porltlve a .pact of th. «on«jflan "J™" " 
ltr ability to taka outrlda crltlclan Into cenrtderation. 

On. point that lr not taken Into c^ld«ratlor. 
hormr. la Sl-an.an'r crltlclw of «"' -n lt ^ f 
guava blar in th. ryrtea. Thlr blar danonrtrater 
in tba dlrtrlbutlon of written and oral 
parlod undar rtudy. Tabla Jjrtijnti both t". *irtrlbu 
tlon and tha dlrtrlbutlon of aarttr in th. f^-year pa 
rlod oatwaan 1980 and 1983 (tha data fron 19M ara not 
rat aval labia). 

Tabla 5. tfrfca. norra H fl d Wff ra.nlti In PL UBfcl l*. 
Hark. S + K 0 «* \l 

taaM U IIS MS U± 

fcchlavap apt aarkn 

1980 <n-62.64B) 2.91 31.11 40.46 23.40 2.12 
i,S? in.l4.329. 3.26 30.87 40.72 23.02 2.12 

1982 (n«65.B95> 3.24 31.32 40. SI 22.75 2. IB 

1983 (n-65.219) 3.18 31.72 40.70 22.35 2.05 
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Written ixuii 
f *H •tuct.nts in 

1980 <n«25.011 

■38.2%) 

1981 (n-26.975 

1982 (n-22.33i 

-35.6*) 
19d3 (n-21.657 
•31.9%} 

°ral euin 

two trpuntUiJ. 

1980 (n-1.263 

■1.9%) 

1981 (n-2.043 

»3.0%) 

1982 (n-1.330 

■l-9%> 

1983 (n-1.264 




3 -40 22.73 

3 -99 ?3.73 

2-95 21.57 

2 -96 23.20 



8-95 41.41 
6.22 48... 
fi - 7 ? 40,98 



43 


.51 


25 


. 75 


4.61 


44 


. 18 


23. 


80 


4.30 


44. 


fe7 


25. 


70 


5.10 


44. 


9fe 


24. 


76 


4.11 



34 


.60 


14.33 


0. 71 


33. 


72 


11.50 


0.34 


36. 


99 


14.96 


0.30 









Th* table w_ . 4 . . 
individual ,tud.nt ..'^'J slt the or lT> 

» nor M i di.trlbution i° nl , t * nd * rdl,ea *e.t. even if 

• Mcrr thp tearhers were a .k.rf Presented 

*» lted lf they think 



9 

:RLC 



stco 



BEST COPY AVAILABLE 



•73- 



th»t each of a mater of trtti la g*iveri the? frig** * emjais- 
■It In the prssent ma system. 

Vable 6. Kelatlve — yhiili on skills end knowledge In 
the present iiar system. 01 teaohere' viewa, ftelatfced 
lmi friouanclti ln«l»l). — 



alternatives 



too 
much 

5 
% 



central ttntemr 
.a dispersion 

too 

little 



-* ampbasls 
2 1 
% % 



-written 15.6 34.4 34,2 11.2 00.$ 3.S17 .t4S 

•mill 

■ kills 00.0 C1.0 31. * 30. f 3*. 3 1.H4 .1*0 

-cultural mm 

knowledge 00.0 09.3 39.4 43.2 11.1 2.3S2 .7*7 

-linguistic 

InoMlt^ 0 0.7 0£.2 70.4 20.* 04.1 3, HP Mi 



Thcrv is a major 4 Iter •fancy N tww tM evmiu- 
ation of writ tan and oral skills in tht tMdwr mate- 
rial, Written skills are generally seem aa baring fcoo 
much emphasis in tha QX a cam, wherea* oral efcllla 
ara ittn as having too llttla emphasis, tn fact more 
than a third of tha tsachers faal that oral akilli ara 
giwn far too llttla enphatla. Tha median la at low at 
1.994, Indicating that thla problem baa considerable 
intensity (3). At this point it teems reasonable to eua- 
pact that at least ona major causs of taachar dlacontant 
hat bain located. 

Tha table alto indicates that the cultural aepects 
of ETL ara given too littls weight. M to llnguiatlc 
knowledgs tha neutral mid category la cheeen by ai 
many at 70.4 per cent of the teachera. In the »oreegian 
curriculum llngulttlc knowledge la dsemphaslied, and in 
recsnt szams It haa not besn tssted in an explicit man- 
nsr. It It interesting to note that so manrr teachers are 
satisfied with the exam in tbis rsspect. The standard 
deviation for this vartabls is loner than for any other 
varlabls we have analytsd so far. In other words thers 
Is both rontldsrabls satlrfactlon and agreement among 
ths teachers on this question. 

Me hare alrsady to some svtent considered the back- 
wash effsct of tie sxam. In a broader perspective dlffs- 
rsnt subtssts mar *>■ evaluated as to thslr motivational 
sfftct on students. In ths national survsy the student 
rsspondsnts wers asked about likes and dlsllkss as to 
ssUcteu test types. 
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r £ ^.^"T^^ -cor din9 to 

g£MUinc« inMlfffi lighted *nd adjusted 



central tendency 

: ; ui.it k . and "■■»'■ 

% * % % % 



Md 



fm"S h ,. 13 - 4 "-a 3o.s i 2 . 3 03. 2 

15-3 33.3 40.3 08.4 01.6 
«->tlon. 08.9 12.1 32.« U .3 15.1 

^r'rltlve ° 8 - 9 23 " 7 36 ' 6 »-7 12.1 

."^ito ry ° 9 - 3 20 - 2 27 - 6 ic. 7 

"."^iptlv. ° 4 -° 113 ".a 32.4 21.5 




•bint, context cf t«chlna In « 

mixed - 
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Tstoia t. WL tMehm' a*aluation if ttfcair awn 
ttactMr training. Hslgfctsd and ad jus tad f rsquancleB 

La^lllli 

altarnatlTes etntral tandancy 

and dlaparsion 





very 










own 


good 




bad 






training 


5 


4 


3 2 1 


m 


i 


tot 


% 


% 


% t % 






!~*oWlsdg« 


39.8 


38.* 
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It It avldsnt that mo it t eschars in tha prasent 
sample in vary wall satisfied with both tha skills 
and knowleUg* aspacts of thalr own training. With tha 
■on pedagogics 1 aspacts of their training, huwaf er , 
tha plctura it dlffarant. Purtharaora thara la a clear 
difference batwaen evaluation and differentiation In 
tha data, evaluation It a ralatlvely mild problaa aftar 
all In this context . Dlffa rant lat Ion It a aueh mora se- 
rious problaa. This result Is Just ona of several In 
ths total surway aaterlal which suggasts that from tha 
teachers ' point-of -view mlxad-ablllty teaching Is a far 
no re serious problaa In EFT than alxad-aMl*ty evalu- 
ation. 



4 Concluding remarks 

I.i tha present papur I heva trfmd to combine data 
f roa a national evaluation syttaa with data f row paoplo 
working In tha systia. I think It la a fair conclusion 
after tan yean of mixed-ability teaching and tasting 
that such eye teat ara quits possible to implement. Soma 
sspacts of thaee syeteas. as notad In this pa par , 
sctually seen to work wall. 

It also seams justified to conclude that whan wa 
choose to sett It for s mlxad-ablllty systaa wa ara not 
tsklng an sasy way out as far ss tha organisation of EFL 
Is concerned. Tha system has Its problems, partlculsry 
In prat* Iril teaching. Thaee ara going to ba s conti- 
nuing challenge for reas-archtrt and teachr - trainers 
for a long time after the implementation cr d -abili- 

ty teaching and ' ng. 

Haastrup <thls seminar) has qi»etti tha con- 

struct vslidlty of the Danish oral exaa lr. relation to 
►he notion of communl *tive competence. This reservation 
is probably justified both foi the orsl and written 
•xfims in Norway, Here, the central problem senes to be 
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rence ( C f. footnote 2). ^ * rM Uk * ««' coh «- 



wd. it i. to ir.i£ct£^h.t "h. llll" " dMl - 

-he teaching guide Vll cLn^. fh. J'f 5 Wltton of 

"™ ^ ass suSi 

*rt«r an, not Haply a question of skill. 



Notes 

1 



L?s^;. o 5.2ssrs ?n^ r t ni t to of ln t. p . 1 t i r <1 

r ?^ ? c *" y out P«rfor««nce analyee. to 

find correspondence. between inon-™.T 

evaluation in different gr.de.. PreliainarY 

in (Even.en (in pre..) and in Even. en (forth-.). 

3 tX d !r n ', b ° th th * "•n-knonn error of cental 
tS^SL ' e ?' °eP* nhel » 1966) and th. fact 
■etli»n of 1.994 i« very low indeed. 
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